Total Jensen divergences: Definition, Properties and k-Means++ Clustering

We present a novel class of divergences induced by a smooth convex function called total Jensendivergences. Those total Jensen divergences are invariant by construction to rotations, a feature yieldingregularization of ordinary Jensen divergences by a conformal factor. We analyze the relationships be-tween this novel class of total Jensen divergences and the recently introduced total Bregman divergences.We then proceed by defining the total Jensen centroids as average distortion minimizers, and study theirrobustness performance to outliers. Finally, we prove that the k-means++ initialization that bypassesexplicit centroid computations is good enough in practice to guarantee probabilistically a constantapproximation factor to the optimal k-means clustering.